REDIRECT: Mapping Drug Prescriptions and Evidence from Biomedical Literature.

To enhance their practice, healthcare professionals need to cross-link various usage recommendations provided by heterogeneous vocabularies that must be retrieved and integrated conjointly. This is the aim of the Knowledge Warehouse / K-Ware platform. It enables establishing relevant bridges between different knowledge sources (structured vocabularies, thesaurus, ontologies) expressed in the semantic web standard languages (i.e. SKOS, OWL, RDF). This poster presents the strategy applied in K-Ware to hide the different aspects of linking literals with medical entities encoded in these knowledge sources to fetch some publications abstracts from Pubmed.

Analysis of 567,758 randomized controlled trials published over 30 years reveals trends in phrases used to discuss results that do not reach statistical significance.

The power of language to modify the reader's perception of interpreting biomedical results cannot be underestimated. Misreporting and misinterpretation are pressing problems in randomized controlled trials (RCT) output. This may be partially related to the statistical significance paradigm used in clinical trials centered around a P value below 0.05 cutoff. Strict use of this P value may lead to strategies of clinical researchers to describe their clinical results with P values approaching but not reaching the threshold to be "almost significant." The question is how phrases expressing nonsignificant results have been reported in RCTs over the past 30 years. To this end, we conducted a quantitative analysis of English full texts containing 567,758 RCTs recorded in PubMed between 1990 and 2020 (81.5% of all published RCTs in PubMed). We determined the exact presence of 505 predefined phrases denoting results that approach but do not cross the line of formal statistical significance (P < 0.05). We modeled temporal trends in phrase data with Bayesian linear regression. Evidence for temporal change was obtained through Bayes factor (BF) analysis. In a randomly sampled subset, the associated P values were manually extracted. We identified 61,741 phrases in 49,134 RCTs indicating almost significant results (8.65%; 95% confidence interval (CI): 8.58% to 8.73%). The overall prevalence of these phrases remained stable over time, with the most prevalent phrases being "marginally significant" (in 7,735 RCTs), "all but significant" (7,015), "a nonsignificant trend" (3,442), "failed to reach statistical significance" (2,578), and "a strong trend" (1,700). The strongest evidence for an increased temporal prevalence was found for "a numerical trend," "a positive trend," "an increasing trend," and "nominally significant." In contrast, the phrases "all but significant," "approaches statistical significance," "did not quite reach statistical significance," "difference was apparent," "failed to reach statistical significance," and "not quite significant" decreased over time. In a random sampled subset of 29,000 phrases, the manually identified and corresponding 11,926 P values, 68,1% ranged between 0.05 and 0.15 (CI: 67. to 69.0; median 0.06). Our results show that RCT reports regularly contain specific phrases describing marginally nonsignificant results to report P values close to but above the dominant 0.05 cutoff. The fact that the prevalence of the phrases remained stable over time indicates that this practice of broadly interpreting P values close to a predefined threshold remains prevalent. To enhance responsible and transparent interpretation of RCT results, researchers, clinicians, reviewers, and editors may reduce the focus on formal statistical significance thresholds and stimulate reporting of P values with corresponding effect sizes and CIs and focus on the clinical relevance of the statistical difference found in RCTs.

Evaluation of reporting quality of randomized controlled trials in patients with COVID-19 using the CONSORT statement.

OBJECTIVE: To evaluate the reporting quality of randomized controlled trials (RCTs) regarding patients with COVID-19 and analyse the influence factors. METHODS: PubMed, Embase, Web of Science and the Cochrane Library databases were searched to collect RCTs regarding patients with COVID-19. The retrieval time was from the inception to December 1, 2020. The CONSORT 2010 statement was used to evaluate the overall reporting quality of these RCTs. RESULTS: 53 RCTs were included. The study showed that the average reporting rate for 37 items in CONSORT checklist was 53.85% with mean overall adherence score of 13.02±3.546 (ranged: 7 to 22). The multivariate linear regression analysis showed the overall adherence score to the CONSORT guideline was associated with journal impact factor (P = 0.006), and endorsement of CONSORT statement (P = 0.014). CONCLUSION: Although many RCTs of COVID-19 have been published in different journals, the overall reporting quality of these articles was suboptimal, it can not provide valid evidence for clinical decision-making and systematic reviews. Therefore, more journals should endorse the CONSORT statement, authors should strictly follow the relevant provisions of the CONSORT guideline when reporting articles. Future RCTs should particularly focus on improvement of detailed reporting in allocation concealment, blinding and estimation of sample size.

Who is blocking access to PubMed? Educational 'Escape Room' for medical residents.

Teaching students how to conduct bibliographic searches in health sciences' databases is essential training. One of the challenges librarians face is how to motivate students during classroom learning. In this article, two hospital libraries, in Spain, used Escape rooms as a method of bringing creativity, teamwork, communication and critical thinking into bibliographic search instruction. Escape rooms are a series of puzzles that must be solved to exit the game. This article explores the methods used for integrating escape rooms into training programmes and evaluates the results. Escape Rooms are a useful tool that can be integrated into residents' training to support their instruction on bibliographic searches. This kind of learning stablishes competences like logical thinking and deductive approaching. These aspects aid participants to make their own decision and to develop social and intellectual skills.

Using Kano diagrams to display the most cited article types, affiliated countries, authors and MeSH terms on spinal surgery in recent 12 years.

BACKGROUND: Citation analysis has been increasingly applied to assess the quantity and quality of scientific research in various fields worldwide. However, these analyses on spinal surgery do not provide visualization of results. This study aims (1) to evaluate the worldwide research citations and publications on spinal surgery and (2) to provide visual representations using Kano diagrams onto the research analysis for spinal surgeons and researchers. METHODS: Article abstracts published between 2007 and 2018 were downloaded from PubMed Central (PMC) in 5 journals, including Spine, European Spine Journal, The Spine Journal, Journal of Neurosurgery: Spine, and Journal of Spinal Disorders and Techniques. The article types, affiliated countries, authors, and Medical subject headings (MeSH terms) were analyzed by the number of article citations using x-index. Choropleth maps and Kano diagrams were applied to present these results. The trends of MeSH terms over the years were plotted and analyzed. RESULTS: A total of 18,808 publications were extracted from the PMC database, and 17,245 were affiliated to countries/areas. The 12-year impact factor for the five spine journals is 5.758. We observed that (1) the largest number of articles on spinal surgery was from North America (6417, 37.21%). Spine earns the highest x-index (= 82.96). Comparative Study has the highest x-index (= 66.74) among all article types. (2) The United States performed exceptionally in x-indexes (= 56.86 and 44.5) on both analyses done on the total 18,808 and the top 100 most cited articles, respectively. The most influential author whose x-index reaches 15.11 was Simon Dagenais from the US. (3) The most cited MeSH term with an x-index of 23.05 was surgery based on the top 100 most cited articles. The most cited article (PMID = 18164449) was written by Dagenais and his colleagues in 2008. The most productive author was Michael G. Fehlings, whose x-index and the author's impact factor are 13.57(= √(13.16*14)) and 9.86(= 331.57/33.64), respectively. CONCLUSIONS: There was a rapidly increasing scientific productivity in the field of spinal surgery in the past 12 years. The US has extraordinary contributions to the publications. Furthermore, China and Japan have increasing numbers of publications on spinal surgery. This study with Kano diagrams provides an insight into the research for spinal surgeons and researchers.

Performance evaluation of three semantic expansions to query PubMed.

BACKGROUND: PubMed is one of the most important basic tools to access medical literature. Semantic query expansion using synonyms can improve retrieval efficacy. OBJECTIVE: The objective was to evaluate the performance of three semantic query expansion strategies. METHODS: Queries were built for forty MeSH descriptors using three semantic expansion strategies (MeSH synonyms, UMLS mappings, and mappings created by the CISMeF team), then sent to PubMed. To evaluate expansion performances for each query, the first twenty citations were selected, and their relevance were judged by three independent evaluators based on the title and abstract. RESULTS: Queries built with the UMLS expansion provided new citations with a slightly higher mean precision (74.19%) than with the CISMeF expansion (70.28%), although the difference was not significant. Inter-rater agreement was 0.28. Results varied greatly depending on the descriptor selected. DISCUSSION: The number of citations retrieved by the three strategies and their precision varied greatly according to the descriptor. This heterogeneity could be explained by the quality of the synonyms. Optimal use of these different expansions would be through various combinations of UMLS and CISMeF intersections or unions. CONCLUSION: Information retrieval tools should propose different semantic expansions depending on the descriptor and the search objectives.

Two Decades of Research Using Taiwan's National Health Insurance Claims Data: Bibliometric and Text Mining Analysis on PubMed.

BACKGROUND: Studies using Taiwan's National Health Insurance (NHI) claims data have expanded rapidly both in quantity and quality during the first decade following the first study published in 2000. However, some of these studies were criticized for being merely data-dredging studies rather than hypothesis-driven. In addition, the use of claims data without the explicit authorization from individual patients has incurred litigation. OBJECTIVE: This study aimed to investigate whether the research output during the second decade after the release of the NHI claims database continues growing, to explore how the emergence of open access mega journals (OAMJs) and lawsuit against the use of this database affect the research topics and publication volume and to discuss the underlying reasons. METHODS: PubMed was used to locate publications based on NHI claims data between 1996 and 2017. Concept extraction using MetaMap was employed to mine research topics from article titles. Research trends were analyzed from various aspects, including publication amount, journals, research topics and types, and cooperation between authors. RESULTS: A total of 4473 articles were identified. A rapid growth in publications was witnessed from 2000 to 2015, followed by a plateau. Diabetes, stroke, and dementia were the top 3 most popular research topics whereas statin therapy, metformin, and Chinese herbal medicine were the most investigated interventions. Approximately one-third of the articles were published in open access journals. Studies with two or more medical conditions, but without any intervention, were the most common study type. Studies of this type tended to be contributed by prolific authors and published in OAMJs. CONCLUSIONS: The growth in publication volume during the second decade after the release of the NHI claims database was different from that during the first decade. OAMJs appeared to provide fertile soil for the rapid growth of research based on NHI claims data, in particular for those studies with two or medical conditions in the article title. A halt in the growth of publication volume was observed after the use of NHI claims data for research purposes had been restricted in response to legal controversy. More efforts are needed to improve the impact of knowledge gained from NHI claims data on medical decisions and policy making.

Protocol registration improves reporting quality of systematic reviews in dentistry.

BACKGROUND: The aims of this study were to assess whether the previous registration of a systematic review (SR) is associated with the improvement of the quality of the report of SRs and whether SR registration reduced outcome reporting bias. METHODS: We performed a search in PubMed for SRs in dentistry indexed in 2017. Data related to SR registration and reporting characteristics were extracted. We analyzed if the reporting of 21 characteristics of included SRs was associated with the prospective registration of protocols or reporting of a previously established protocol. The association between prospective registering of protocols, reporting of funding and number of included studies versus outcome reporting bias was tested via multivariable logistic regression. RESULTS: We included 495 SRs. One hundred and 62 (32.7%) SRs reported registering the SR protocol or working from a previously established protocol. Thirteen reporting characteristics were described statistically significant in SRs registered versus SRs that were not. Publication bias assessment and Report the number of participants showed the highest effects favoring the register (RR 1.59, CI 95% 1.19-2.12; RR 1.58, CI 95% 1.31-1.92 respectively). Moreover, Registration was not significantly linked with the articles' reporting statistical significance (OR 0.96, CI 95% 0.49-1.90). CONCLUSION: There is a positive influence of previously registering a protocol in the final report quality of SRs in dentistry. However, we did not observe an association between protocol registration and reduction in outcome reporting bias.

Systematic Evaluation of Research Progress on Natural Language Processing in Medicine Over the Past 20 Years: Bibliometric Study on PubMed.

BACKGROUND: Natural language processing (NLP) is an important traditional field in computer science, but its application in medical research has faced many challenges. With the extensive digitalization of medical information globally and increasing importance of understanding and mining big data in the medical field, NLP is becoming more crucial. OBJECTIVE: The goal of the research was to perform a systematic review on the use of NLP in medical research with the aim of understanding the global progress on NLP research outcomes, content, methods, and study groups involved. METHODS: A systematic review was conducted using the PubMed database as a search platform. All published studies on the application of NLP in medicine (except biomedicine) during the 20 years between 1999 and 2018 were retrieved. The data obtained from these published studies were cleaned and structured. Excel (Microsoft Corp) and VOSviewer (Nees Jan van Eck and Ludo Waltman) were used to perform bibliometric analysis of publication trends, author orders, countries, institutions, collaboration relationships, research hot spots, diseases studied, and research methods. RESULTS: A total of 3498 articles were obtained during initial screening, and 2336 articles were found to meet the study criteria after manual screening. The number of publications increased every year, with a significant growth after 2012 (number of publications ranged from 148 to a maximum of 302 annually). The United States has occupied the leading position since the inception of the field, with the largest number of articles published. The United States contributed to 63.01% (1472/2336) of all publications, followed by France (5.44%, 127/2336) and the United Kingdom (3.51%, 82/2336). The author with the largest number of articles published was Hongfang Liu (70), while Stéphane Meystre (17) and Hua Xu (33) published the largest number of articles as the first and corresponding authors. Among the first author's affiliation institution, Columbia University published the largest number of articles, accounting for 4.54% (106/2336) of the total. Specifically, approximately one-fifth (17.68%, 413/2336) of the articles involved research on specific diseases, and the subject areas primarily focused on mental illness (16.46%, 68/413), breast cancer (5.81%, 24/413), and pneumonia (4.12%, 17/413). CONCLUSIONS: NLP is in a period of robust development in the medical field, with an average of approximately 100 publications annually. Electronic medical records were the most used research materials, but social media such as Twitter have become important research materials since 2015. Cancer (24.94%, 103/413) was the most common subject area in NLP-assisted medical research on diseases, with breast cancers (23.30%, 24/103) and lung cancers (14.56%, 15/103) accounting for the highest proportions of studies. Columbia University and the talents trained therein were the most active and prolific research forces on NLP in the medical field.

Comparing Bibliometric Analysis Using PubMed, Scopus, and Web of Science Databases.

Literature databases (i.e., PubMed, Scopus, and Web of Science) differ in terms of their coverage, focus, and the tool they provide. PubMed focuses mainly on life sciences and biomedical disciplines, whereas Scopus and Web of Science are multidisciplinary. The protocol described in the current study was used to search for publications from Jordanian authors in the years 2013-2017. In this protocol, how to use each database to conduct this type of search is explained in detail. A Scopus search resulted in the highest number of documents (11,444 documents), followed by a Web of Science search (10,943 documents). PubMed resulted in a smaller number of documents due to its narrower scope and coverage (4,363 documents). The results also show a yearly trend in: (1) the number of publications, (2) the disciplines that have the most publications, (3) the countries of collaboration, and (4) the number of open access publications. In contrast, PubMed has a sophisticated keyword optimization service (i.e., Medical Subject Heading, or MeSH), while both Scopus and Web of Science provide search analysis tools that can produce representative figures. Finally, the features of each database are explained in detail and several indices that can be extracted using the search results are provided. This study provides a base for using literature databases for bibliometric analysis.

Single screening versus conventional double screening for study selection in systematic reviews: a methodological systematic review.

BACKGROUND: Stringent requirements exist regarding the transparency of the study selection process and the reliability of results. A 2-step selection process is generally recommended; this is conducted by 2 reviewers independently of each other (conventional double-screening). However, the approach is resource intensive, which can be a problem, as systematic reviews generally need to be completed within a defined period with a limited budget. The aim of the following methodological systematic review was to analyse the evidence available on whether single screening is equivalent to double screening in the screening process conducted in systematic reviews. METHODS: We searched Medline, PubMed and the Cochrane Methodology Register (last search 10/2018). We also used supplementary search techniques and sources ("similar articles" function in PubMed, conference abstracts and reference lists). We included all evaluations comparing single with double screening. Data were summarized in a structured, narrative way. RESULTS: The 4 evaluations included investigated a total of 23 single screenings (12 sets for screening involving 9 reviewers). The median proportion of missed studies was 5% (range 0 to 58%). The median proportion of missed studies was 3% for the 6 experienced reviewers (range: 0 to 21%) and 13% for the 3 reviewers with less experience (range: 0 to 58%). The impact of missing studies on the findings of meta-analyses had been reported in 2 evaluations for 7 single screenings including a total of 18,148 references. In 3 of these 7 single screenings - all conducted by the same reviewer (with less experience) - the findings would have changed substantially. The remaining 4 of these 7 screenings were conducted by experienced reviewers and the missing studies had no impact or a negligible on the findings of the meta-analyses. CONCLUSIONS: Single screening of the titles and abstracts of studies retrieved in bibliographic searches is not equivalent to double screening, as substantially more studies are missed. However, in our opinion such an approach could still represent an appropriate methodological shortcut in rapid reviews, as long as it is conducted by an experienced reviewer. Further research on single screening is required, for instance, regarding factors influencing the number of studies missed.

PubMed coverage varied across specialties and over time: a large-scale study of included studies in Cochrane reviews.

OBJECTIVE: PubMed is one of the most commonly used search tools in biomedical and life sciences. Existing studies of database coverage generally conclude that searching PubMed may not be sufficient although some find that the contributions from other databases are modest at best. However, generalizability of the studies of the coverage of PubMed is typically restricted. The objective of this study is to analyze the coverage of PubMed across specialties and over time. STUDY DESIGN AND SETTING: We use the more than 50,000 included studies in all Cochrane reviews published from 2012 to 2016 as our population and examine if the studies and resulting publications can be identified in PubMed. RESULTS: The results show that PubMed has a coverage of 70.9, 95% confidence interval (CI) (68.40, 73.30) of all the included publications and 82.8%, 95% CI (80.9, 84.7) of the included studies. There are huge differences in coverage across and within specialties. In addition, coverage varies within groups over time. CONCLUSION: Databases used for searching topics within the groups with highly varying or low coverage should be chosen with care as PubMed may have a relatively low coverage.

Exploring PubMed as a reliable resource for scholarly communications services.

OBJECTIVE: PubMed's provision of MEDLINE and other National Library of Medicine (NLM) resources has made it one of the most widely accessible biomedical resources globally. The growth of PubMed Central (PMC) and public access mandates have affected PubMed's composition. The authors tested recent claims that content in PMC is of low quality and affects PubMed's reliability, while exploring PubMed's role in the current scholarly communications landscape. METHODS: The percentage of MEDLINE-indexed records was assessed in PubMed and various subsets of records from PMC. Data were retrieved via the National Center for Biotechnology Information (NCBI) interface, and follow-up interviews with a PMC external reviewer and staff at NLM were conducted. RESULTS: Almost all PubMed content (91%) is indexed in MEDLINE; however, since the launch of PMC, the percentage of PubMed records indexed in MEDLINE has slowly decreased. This trend is the result of an increase in PMC content from journals that are not indexed in MEDLINE and not a result of author manuscripts submitted to PMC in compliance with public access policies. Author manuscripts in PMC continue to be published in MEDLINE-indexed journals at a high rate (85%). The interviewees clarified the difference between the sources, with MEDLINE serving as a highly selective index of journals in biomedical literature and PMC serving as an open archive of quality biomedical and life sciences literature and a repository of funded research. CONCLUSION: The differing scopes of PMC and MEDLINE will likely continue to affect their overlap; however, quality control exists in the maintenance and facilitation of both resources, and funding from major grantors is a major component of quality assurance in PMC.

Predatory publications in evidence syntheses.

OBJECTIVES: The number of predatory journals is increasing in the scholarly communication realm. These journals use questionable business practices, minimal or no peer review, or limited editorial oversight and, thus, publish articles below a minimally accepted standard of quality. These publications have the potential to alter the results of knowledge syntheses. The objective of this study was to determine the degree to which articles published by a major predatory publisher in the health and biomedical sciences are cited in systematic reviews. METHODS: The authors downloaded citations of articles published by a known predatory publisher. Using forward reference searching in Google Scholar, we examined whether these publications were cited in systematic reviews. RESULTS: The selected predatory publisher published 459 journals in the health and biomedical sciences. Sixty-two of these journal titles had published a total of 120 articles that were cited by at least 1 systematic review, with a total of 157 systematic reviews citing an article from 1 of these predatory journals. DISCUSSION: Systematic review authors should be vigilant for predatory journals that can appear to be legitimate. To reduce the risk of including articles from predatory journals in knowledge syntheses, systematic reviewers should use a checklist to ensure a measure of quality control for included papers and be aware that Google Scholar and PubMed do not provide the same level of quality control as other bibliographic databases.

Bibliometrics and French healthcare institutions from 2004 to 2014.

INTRODUCTION: Research activity evaluation in French hospitals is based on the number of publications, author position (first, second, third, second-to-last, last, investigator list, and "Other") and journal category (A being the highest category followed by B, C, D, E, and NC). METHODS: The profile of publications over the 2004-2014 period in terms of these indicators was evaluated. Hospitals were classified into six groups according to administrative status. Time trends were analysed by three models. One-way ANOVA followed by Tukey's test was performed. RESULTS: A total of 192886 publications were analysed. The increase in the number of publications ranged from 628% for for-profit private hospitals to 141% for public teaching hospitals. The most frequent category was B for cancer centres (25%), whereas this was E in public teaching (22%) and non-teaching hospitals (28%), in not-for-profit private hospitals (25%), in the military hospital (30%), and in for-profit private hospitals (24%). The first position was the most frequent for public teaching hospitals (38%) and the military hospital (44%), whereas the "Other" position was the most frequent in cancer centres (26%), in public non-teaching hospitals (28%), in not-for-profit private hospitals (27%), and in for-profit private hospitals (29%). DISCUSSION: Different patterns were identified. The author position indicated that all types of hospital are involved in research projects. This study also found that public non-teaching hospitals, not-for-profit private hospitals, for-profit private hospitals, and cancer centres collaborated with other institutions which were often distinguished by publishing in high-category journals.

Fast and scalable neural embedding models for biomedical sentence classification.

BACKGROUND: Biomedical literature is expanding rapidly, and tools that help locate information of interest are needed. To this end, a multitude of different approaches for classifying sentences in biomedical publications according to their coarse semantic and rhetoric categories (e.g., Background, Methods, Results, Conclusions) have been devised, with recent state-of-the-art results reported for a complex deep learning model. Recent evidence showed that shallow and wide neural models such as fastText can provide results that are competitive or superior to complex deep learning models while requiring drastically lower training times and having better scalability. We analyze the efficacy of the fastText model in the classification of biomedical sentences in the PubMed 200k RCT benchmark, and introduce a simple pre-processing step that enables the application of fastText on sentence sequences. Furthermore, we explore the utility of two unsupervised pre-training approaches in scenarios where labeled training data are limited. RESULTS: Our fastText-based methodology yields a state-of-the-art F1 score of.917 on the PubMed 200k benchmark when sentence ordering is taken into account, with a training time of only 73 s on standard hardware. Applying fastText on single sentences, without taking sentence ordering into account, yielded an F1 score of.852 (training time 13 s). Unsupervised pre-training of N-gram vectors greatly improved the results for small training set sizes, with an increase of F1 score of.21 to.74 when trained on only 1000 randomly picked sentences without taking sentence ordering into account. CONCLUSIONS: Because of it's ease of use and performance, fastText should be among the first choices of tools when tackling biomedical text classification problems with large corpora. Unsupervised pre-training of N-gram vectors on domain-specific corpora also makes it possible to apply fastText when labeled training data are limited.

Study filters for non-randomized studies of interventions consistently lacked sensitivity upon external validation.

BACKGROUND: Little evidence is available on searches for non-randomized studies (NRS) in bibliographic databases within the framework of systematic reviews. For instance, it is currently unclear whether, when searching for NRS, effective restriction of the search strategy to certain study types is possible. The following challenges need to be considered: 1) For non-randomized controlled trials (NRCTs): whether they can be identified by established filters for randomized controlled trials (RCTs). 2) For other NRS types (such as cohort studies): whether study filters exist for each study type and, if so, which performance measures they have. The aims of the present analysis were to identify and validate existing NRS filters in MEDLINE as well as to evaluate established RCT filters using a set of MEDLINE citations. METHODS: Our analysis is a retrospective analysis of study filters based on MEDLINE citations of NRS from Cochrane reviews. In a first step we identified existing NRS filters. For the generation of the reference set, we screened Cochrane reviews evaluating NRS, which covered a broad range of study types. The citations of the studies included in the Cochrane reviews were identified via the reviews' bibliographies and the corresponding PubMed identification numbers (PMIDs) were extracted from PubMed. Random samples comprising up to 200 citations (i.e. 200 PMIDs) each were created for each study type to generate the test sets. RESULTS: A total of 271 Cochrane reviews from 41 different Cochrane groups were eligible for data extraction. We identified 14 NRS filters published since 2001. The study filters generated between 660,000 and 9.5 million hits in MEDLINE. Most filters covered several study types. The reference set included 2890 publications classified as NRS for the generation of the test sets. Twelve test sets were generated (one for each study type), of which 8 included 200 citations each. None of the study filters achieved sufficient sensitivity (≥ 92%) for all of the study types targeted. CONCLUSIONS: The performance of current NRS filters is insufficient for effective use in daily practice. It is therefore necessary to develop new strategies (e.g. new NRS filters in combination with other search techniques). The challenges related to NRS should be taken into account.

Systematic review of the psychometric properties of instruments to measure sexual desire.

BACKGROUND: Sexual desire is one of the domains of sexual function with multiple dimensions, which commonly affects men and women around the world. Classically, its assessment has been applied through self-report tools; however, an issue is related to the evidence level of these questionnaires and their validity. Therefore, a systematic review addressing the available questionnaires is really relevant, since it will be able to show their psychometric properties and evidence levels. METHOD: A systematic review was carried out in the PubMed, EMBASE, PsycINFO, Science Direct, and Web of Science databases. The search strategy was developed according to the following research question and combination of descriptors and keywords, including original studies with no limit of publication date and in Portuguese, English, and Spanish. Two reviewers carried out the selection of articles by abstracts and full texts as well as the analysis of the studies independently. The methodological quality of the instruments was evaluated by the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) checklist. RESULTS: The search resulted in 1203 articles, of which 15 were included in the review. It identified 10 instruments originally developed in the English language. Unsatisfactory results on methodological quality were evidenced in cultural adaptation studies with no description of the steps of the processes and inadequacy of techniques and parameters of adequacy for models. The Principal Component Analysis with Varimax rotation predominated in the studies. CONCLUSIONS: The limitation of the techniques applied in the validation process of the reviewed instruments was evident. A limitation was observed in the number of adaptations conducted and contexts to which the instruments were applied, making it impossible to reach a better understanding of the functioning of instruments. In future studies, the use of robust techniques can ensure the quality of the psychometric properties and the accuracy and stability of instruments. A detailed description of procedures and results in validation studies may facilitate the selection and use of instruments in the academic and/or clinical settings. SYSTEMATIC REVIEW REGISTRATION: PROSPERO CRD42018085706.

A Deep Learning Method to Automatically Identify Reports of Scientifically Rigorous Clinical Research from the Biomedical Literature: Comparative Analytic Study.

BACKGROUND: A major barrier to the practice of evidence-based medicine is efficiently finding scientifically sound studies on a given clinical topic. OBJECTIVE: To investigate a deep learning approach to retrieve scientifically sound treatment studies from the biomedical literature. METHODS: We trained a Convolutional Neural Network using a noisy dataset of 403,216 PubMed citations with title and abstract as features. The deep learning model was compared with state-of-the-art search filters, such as PubMed's Clinical Query Broad treatment filter, McMaster's textword search strategy (no Medical Subject Heading, MeSH, terms), and Clinical Query Balanced treatment filter. A previously annotated dataset (Clinical Hedges) was used as the gold standard. RESULTS: The deep learning model obtained significantly lower recall than the Clinical Queries Broad treatment filter (96.9% vs 98.4%; P<.001); and equivalent recall to McMaster's textword search (96.9% vs 97.1%; P=.57) and Clinical Queries Balanced filter (96.9% vs 97.0%; P=.63). Deep learning obtained significantly higher precision than the Clinical Queries Broad filter (34.6% vs 22.4%; P<.001) and McMaster's textword search (34.6% vs 11.8%; P<.001), but was significantly lower than the Clinical Queries Balanced filter (34.6% vs 40.9%; P<.001). CONCLUSIONS: Deep learning performed well compared to state-of-the-art search filters, especially when citations were not indexed. Unlike previous machine learning approaches, the proposed deep learning model does not require feature engineering, or time-sensitive or proprietary features, such as MeSH terms and bibliometrics. Deep learning is a promising approach to identifying reports of scientifically rigorous clinical research. Further work is needed to optimize the deep learning model and to assess generalizability to other areas, such as diagnosis, etiology, and prognosis.